JOURNAL OF LEARNING ANALYTICS 

(2014). A Longitudinal Study on Learner Career Advancement in MOOCs. Journal of Learning Analytics, 1 (3), 203-206. 


S-LAR 


A Longitudinal Study on Learner Career Advancement in MOOCs 


Yuan Wang, Luc Paquette, and Ryan Baker 

Columbia University Teachers College, USA 
elle.wang@columbia.edu 

ABSTRACT: In this paper, we present progress towards a longitudinal study of the post-course 
career advancement of MOOC learners. We present initial results and analysis plans for how to 
link this to in-course behaviour, towards better understanding the goals of all MOOC learners. 
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1. INTRODUCTION 


MOOCs, massive open online courses, have garnered worldwide attention as a new platform for 
learning over the past three years. A growing community of researchers from various disciplines have 
studied engagement patterns in MOOC (e.g., Kizilcec, Piech, & Schneider, 2013; Clow, 2013), and found 
out MOOC learners exhibited highly varied ways of interacting with and using the courses they enroll in. 
Unlike in traditional online learning platforms, many MOOC learners do not consider completing a 
course their primary goal (Fini, 2009; Belanger & Thornton, 2013). Although the dropout rate is very 
high, it is generally agreed that completion rate in the context of MOOCs cannot be easily equated with 
previous generations of learning platforms (cf. DeWaard et al., 2011). 

As such, challenges have been raised on how to define learner success in the context of MOOCs (e.g., 
Breslow et al. 2013). This question is pertinent in that traditional metrics assessing the success of online 
learning may not be relevant to the needs and goals of MOOC learners. Existing research has focused on 
two dimensions in addressing this question: achievement and persistence. First, some studies of learner 
success have examined metrics mirroring a traditional classroom, such as grades of quizzes and exams 
(cf. Belanger & Thornton, 2013; MOOC @ Edinburgh, 2013). Second, studies linking success with 
persistence focused on learner usage and participation with course components such as videos (e.g., 
Guo, Kim, & Rubin, 2014) and discussion forums (e.g., Yang, Sinha, David, & Rose, 2013). 

Since data on MOOC learner information is highly fragmented and diffuse (McAulay, Stewart, & 
Siemens, 2010), it may be useful to incorporate data beyond the immediate course platform and the 
pre-designated time frame for a specific course in the efforts of better understanding learner behaviour. 
However, little has been investigated in terms of how a MOOC has impacted a student beyond 1) the 
duration of the course prescribed by the instructor or the design of the course platform and 2) the 
context of the course platform itself. Therefore, we propose an inclusive plan to study student success, 
going beyond interactions with the course itself, but also by taking into account data collected beyond 
the constraints of the course and course platform. Specifically, we will investigate the effects of course 
participation on longitudinal data reflecting participant career advancement after taking the MOOC. 
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2. METHOD 
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The proposed research is being carried out within the context of a MOOC, "Big Data in Education," 
delivered via Coursera in Fall 2013. As introduced in the previous section, we plan to integrate course 
data with longitudinal data in order to better understand and assess learner success in the context of 
MOOCs. To this end, we have obtained multiple data sources. First, achievement during the course is 
represented by course grade data on student performance on eight weekly assignments. Second, 
persistence is represented by log data capturing learner behaviour interacting with course components 
together with text data extracted from the discussion forums. These data were obtained in a raw form, 
and much feature engineering will be required. In addition, variables extracted from an early course 
motivational survey, measuring student academic efficacy and goal orientation, and an exit survey, 
measuring student intent on future direction after the course, have been collected. Third, for 
community participation, we have started to look into career advancement and learner participation in 
communities of practice after taking the MOOC. This can be measured in several ways, including looking 
at whether students choose to join relevant societies, such as the International Educational Data Mining 
Society, attended conferences, such as the Learning Analytics and Knowledge Conference and the 
International Conference on Educational Data Mining, and published in relevant fields. In addition, social 
media participation will be studied. We will then develop predictive models that infer longitudinal 
participation from interactions within the course. 

3. RESULTS AND DISCUSSION 

The exit survey indicated that 430 out of 536 □ more than 

intended skills of this MOOC in their career. Going forward from this to community participation, we 
have found that 35 students who registered for the MOOC joined the International Educational Data 
Mining Society after the course started. Preliminary analyses indicated that these 35 students 
disproportionately completed the course; 20% of students who joined the society completed the course, 
whereas only 1.3% of the remaining students did (x2(l) = 97.438, p < 0.001). Although 35 is a small 
number in comparison to the total enrollment, it only reflects one small aspect of community 
participation. Further learner participation data will also be collected. 

The present study sets the goal of investigating MOOC impact and learner success longitudinally by 
using course data to predict career-related learner decisions and trajectories after the course finishes. 

This project therefore has the goal of investigating MOOC learner success beyond traditional indicators 
of successful course completion. This study is aimed toward understanding what aspects in the use of 
MOOCs are predictive of eventual public participation in the community of practice. As such, if this 
project is successful, it may contribute to our understanding of what it means for a MOOC to succeed, 
and how the effectiveness of a MOOC can be assessed more comprehensively. 
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